Add list_transform - #10503

## Summary

Adds `ListArray::compact` and a list view rebuild mode,
`ListViewRebuildMode::MakeExactWithEmptyNulls`. Both return the same
lists over only the elements that valid lists reference: nothing before
the first list, after the last, between lists or under a null list.

A list column can hold elements that no valid list shows. Slicing keeps
the parent's elements on both sides, and a null list may still cover
elements. A function that only reads values does not care, but a
function that can fail on an element, such as a checked `x + 1`, must
not see them: an overflow in an element under a null list would fail the
whole batch. `list_transform` (#10503, stacked on this PR) is the first
user; it applies its lambda to the rebuilt elements whenever the lambda
can fail.

For example:

```text
elements  [9, 1, 2, 9, 9, 3]
offsets   [1, 3, 5, 6]
validity  [valid, null, valid]
lists     [1, 2], null over [9, 9], [3]

rebuilt
elements  [1, 2, 3]
offsets   [0, 2, 2, 3]
```

## How it works

**`ListArray`**

- Non-nullable or all-valid lists with nothing at the ends are returned
as they are, after reading two offsets.
- Otherwise it looks for null lists that hold elements: one pass marks
the lists whose next offset is larger, and one bit operation clears the
valid ones. Only when bits are left does it collect them into runs.
- With no such run, only the ends go: the elements are sliced to `[first
offset, last offset)` and the offsets are shifted down, without copying
an element.
- With some, the elements between those runs are kept, by a slice if one
range is left and otherwise by one filter whose mask carries the ranges,
so it copies range by range. The offsets are copied once and each one
moves down by the elements dropped before it, so the null lists in those
runs become empty.
- The offsets keep their type and the validity is unchanged.

**`ListViewRebuildMode::MakeExactWithEmptyNulls`**

- `MakeExact` only trims the ends of a view that is already zero-copy to
a list, so its null lists keep their elements. The new mode drops those
too.
- Such a view has its lists back to back, so it goes through the same
two steps on its offsets, with the last list's end as the final
boundary. It keeps its layout and its offsets' and sizes' types. Only
the sizes of the null lists that held elements are set to 0; otherwise
the sizes are kept as they are.
- Any other view is rebuilt with `MakeExact`, which already copies only
the valid lists' elements and leaves null lists empty. `MakeExact`'s
documentation now says both.
- The result is always zero-copy to a list.

## Benchmark

`list_transform` with a checked `x + 1` over 100k lists (~800k `i32`
elements, ~1/8 of the lists null), through #10503, exported to Arrow,
median µs over three alternating runs of a local divan benchmark.
"Before" is #10503 without this PR, which used the existing list view
rebuild instead.

| Lists | Before | This PR | DataFusion 54.1 `array_transform` |
|---|---|---|---|
| Null lists holding elements | 1,272 | 253 | 224 |
| Null lists already empty | 97 | 73 | 60 |
| Sliced, with elements on both sides | 78 | 81 | 55 |

## Testing

- `rstest` cases for both types: ends to drop, null lists holding
elements or not, every list null, no lists, an empty valid list between
null lists, overlapping and out-of-order views, views with and without
the zero-copy flag.
- Exact arrays come back as the same arrays (`ArrayRef::ptr_eq`), and a
sliced `ListArray` whose validity starts mid-byte is rebuilt correctly.
- A seeded random test compares `ListArray` and both kinds of view
against a reference that gathers each valid list's elements, over five
offset types, with non-nullable validity and validity arrays that start
mid-byte.

---------

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

## Summary

Closes #9

Spark asks the scan for only the nested fields a query uses, but the
connector read every nested column whole. `SELECT s.b` decoded all of
`s`.

Struct columns are now pruned all the way down to the native scan, at
any depth. Fields inside arrays and maps are still read whole; that
needs `list_transform`
([vortex-data#10503](vortex-data#10503)).

## Changes

**Spark connector**
- `pruneColumns` keeps Spark's pruned type for struct columns.
- `NestedSchemaPruning` turns the pruned read schema into a native
projection:
- `select([b], $.s)` when only leaf fields are pruned. This keeps the
struct's own nulls.
- `mask(pack(..), is_not_null($.s))` when a child struct is pruned too.
`pack` has no nulls, so the mask carries them over. Non-nullable structs
skip the mask.

**JNI**
- Adds an `Expression.mask` binding.

**`StructReader`**
- No changes. Upstream
[vortex-data#10536](vortex-data#10536)
collapses single-field partitions, so the Spark projections are read
field by field without a special case. An earlier version of this PR
extended `StructReader`'s direct projection path, which vortex-data#10536 removed.

**Tests**
- Spark: null structs, null leaves, nulls at each level of a struct
inside a struct, and a pushed filter on a field that was pruned away.
- Layout: segment-count tests, now run against vortex-data#10536's partitioned
path. A pruned nullable struct column reads only its kept fields plus
its validity.
- `CountingSegmentSource` now lives in `segments/test.rs`, replacing
three copies.

## Benchmark

Measured before the merge of vortex-data#10536. JMH, 2 forks × 10 iterations,
release native library, Spark 4.1 local[4]. The table has 2M rows, with
a nullable 20-field struct `s` and the same struct nested at
`outer.inner`. `nestedSchemaPruning=false` is the old behaviour, where
the whole struct is read.

| Query | Pruning on | Pruning off |
|---|---|---|
| `max(s.f3)` | 26.2 ± 2.3 ms | 40.2 ± 3.4 ms |
| `max(outer.inner.f3)` | 22.0 ± 1.1 ms | 37.1 ± 2.3 ms |

## Checks

- `cargo test -p vortex-layout --lib`: 696 passed
- `cargo clippy -p vortex-layout --all-targets --all-features -- -D
warnings`: clean
- `cargo +nightly-2026-09-10 fmt --check -p vortex-layout -p
vortex-jni`: clean
- `./gradlew spotlessCheck :vortex-jni:test :vortex-spark_2.13:test
:vortex-spark_2.12:test`: all pass (JNI 97, Spark 2.13 225, Spark 2.12
225 with 2 Spark-4-only tests skipped).
- Not run since vortex-data#10536: the benchmark, the full workspace suite.

## Known gaps

- `Expression` factories don't keep their inputs reachable while the
native call runs. In theory GC can free a temporary first. This affects
every factory, not just the new `mask`, so it's left for a separate fix.
- `count(*)`, or a query that selects only partition columns, still
reads every column. This predates the PR.
- Pruning two levels inside a nullable struct reads the whole inner
struct. For `nullable_outer.inner.c`, `inner.a` and `inner.b` are still
read; sibling fields of `inner` are skipped. The cause is upstream:
`StructReader` masks `$.inner` with the parent's validity before the
`select`, so the `select` can't be pushed down into `inner`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>